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Washington  University  Hunan  Resources  Research  Office.  HumRRO'a 
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under  contract  with  the  Department  of  the  Army  is  to  conduct  research 
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Prefatory  Net* 

This  paper  is  based  on  research  performed  by  the  Human  Resources 
Research  Organization,  Division  No.  1  (System  Operations)  at  Alexandria, 
Virginia,  under  Work  Unit  IMPACT,  Prototypes  of  Computerized  Training 
for  Army  Personnel. 

The  paper  was  presented  by  Dr.  Seidel  at  a  conference  titled 
‘Application  of  Computers  to  Training*  sponsored  by  the  National  Security 
Industrial  Association  (NSIA)  in  February  1970. 

An  earlier  version  of  this  paper  was  presented  by  Dr.  Seidel  at  the 
National  Council  for  Educational  Technology  (NCET),  Computer  Based 
Learning  Seminar,  University  of  Leeds,  Leeds,  England,  September  1969- 


RESOURCE  ALLOCATIONS  TO  EFFECT  OPERATIONALLY  USEFUL  CAI 


Robert  J.  Seidel  and  Felix  F.  Kopstein 

INTRODUCTION 

The  broad  picture  of  resource  allocation  for  research  and  development 
(R&D)  and  implementation  of  computer-administered  (or  computer-assisted) 
instruction  (CAI)  will  be  discussed  in  this  paper.  Specific  cost 
projections  within  an  operational  setting  have  been  discussed  previously 
(Kopstein  and  Seidel,  1)  and  therefore  will  be  dealt  with  only  briefly. 
"Resource  allocation"  will  refer  to  funds,  people,  facilities,  and  the 
delegation  of  appropriate  authority  to  formulate  appropriate  policy. 

\nd  the  fact  that  all  these  many  resources — government,  industry,  and 
education  will  be  considered — means  that  inevitably  politics  and 
economics  will  be  intertwined  in  the  subsequent  discussion. 

The  features  of  this  paper  will  include  a  brief  description  and 
justification  for  CAI  as  a  technology.  The  need  for  incorporating  a 
systems  approach  to  educational  innovation  will  follow  to  set  the 
proper  framework  for  the  magnitude  and  complexity  of  the  required 
research  and  development  effort.  An  analogue  to  a  corporate  level 
of  investment  in  research  and  development,  3%  to  5%  of  income,  is 
proposed  to  effect  the  orderly  transition  of  CAI  from  "breadboard," 
through  prototype,  to  an  operational  system. 

Problems  that  arise  in  the  course  of  considering  appropriate  resource 
allocation  stem  in  part  from  the  fact  that  none  of  the  so-called  enti¬ 
ties — government,  industry,  or  education— are  monolithic.  Government 
involvement  will  be  both  central  and  local.  Industry  in  CAI  includes 
hardware  manufacturers,  book  companies,  and  so  forth.  Education 
includes  central  and  regional  administrators,  teachers,  research  and 
development  personnel,  as  well  as  the  ultimate  user,  the  students 
Problems  of  program  management  <ind  coordination  are  thereby  made 
difficult. 

Finally,  a  partnership  model  will  be  proposed  to  evolve  operational 
CAI.  The  vehicle  of  a  national  R$D  center  with  regional  satellites  is 
suggested  both  to  provide  proper  training  and  to  permit  flexibility  of 
research  approaches  to  accomplish  the  goal  of  operational  CAI.  These 
points  are  discussed  in  the  context  that:  "anything  worth  doing  is 
worth  doing  right." 

Thus,  we  will  ask  if  research  and  development  into  CAI  is  worth 
expenditure  of  money,  time,  and  effort.  As  you  might  predict,  our 
answer  will  be  affirmative.  Granted  this  is  so,  what  expenditures  are 
necessary  to  "do  it  right"?  Finally,  how  should  the  resources  be 
properly  allocated  among  industry,  government,  and  the  educational 
community  to  bring  CAI  to  fruition?  Fruition  in  this  case  refers  to 
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the  actual  production  of  multiple  copies  of  operational  CAI  systems 
usable  in  schools  and  universities  throughout  a  nation. 

THE  FRAMEWORK 


CAI— A  Necessary  Technology 

Various  prominent  individuals  and  agencies  in  the  United  States 
have  indicated  the  need  and  desirability  of  making  use  of  the  educa¬ 
tional  opportunities  inherent  in  individualized  instruction.  The 
Committee  for  Economic  Development  has  represented  the  broadest  cross- 
section  of  U.S.  society  in  its  appraisal  (200  businessmen,  educators, 
psychologists,  and  community  leaders).  In  essence,  they  present  a 
convincing  case  for  a  failure  of  our  current  educational  system  to  take 
advantage  of  modern  technology  and  to  deal  effectively  with  the  increas¬ 
ing  requirements  of  our  complex  society.  The  Committee  has  pointed 
out  that  "individualized  instruction  geared  to  the  individual  interests, 
abilities  and  learning  rate  is  one  of  the  cherished  goals  of  American 
education.  It  is  an  aspiration  which  we  wholeheartedly  share,  yet  the 
schools  are  making  very  slow  headway  in  this  direction  through  present 
means  ..."  (2).  Psychologists  have  been  engaged  in  the  study  of 
individual  differences  for  years.  Teachers  have  long  been  complaining 
of  the  inability— administratively — of  coping  with  students  as  indi¬ 
viduals  and  have  been  thereby  forced  to  teach,  for  the  most  part,  to 
the  mean  of  a  class. 

Our  view  is  that  CAI  is  the  leading,  operationally  defined  edge  of 
a  model  of  individualized  instruction.  It  represents  the  potential 
(with  all  due  respect  to  objection  by  Oettinger  and  Marks  (3))  for  a 
quantum  leap  in  adapting  instruction  to  the  momentary  needs  and  capa¬ 
bilities  of  the  individual  student.  It  provides  the  basis  for  an 
iteratively  improving  instructional  environment. 

A  word  should  be  added  here  about  the  distinction  between  technique 
and  technology.  The  criteria  for  evaluating  them  differ.  One  technique 
may  be  properly  compared  to  another  within  the  same  system  model. 

Teaching  reading  by  the  phonics  method  versus  "look  and  say"  is  an 
example  of  such  a  comparison.  Indeed,  studies  have  been  conducted 
attesting  to  hold  all  other  components  of  the  momentary  education 
system  constant  save  the  difference  in  technique. 

In  evaluating  a  new  technology,  however,  such  relatively  clear  and 
simplistic  comparisons  are  insufficient.  It  ^<ay  well  happen,  as  with 
CAI  and  individualized  instruction  in  general,  that  the  system  in 
question  must  be  redefined.  It  is  silly  to  mold  a  computerized, 
individualized  teaching  environment  around  a  six-hour  day.  It  is  also 
improper  to  consider  the  economics  of  the  new  technology  in  terms  that 
neglect  the  new  opportunities  opened  up  and  hitherto  impossible.  To 
quote  Harley,  "When  economies  result  [from  using  the  new  technology] 
they  are  derived  from  the  reduced  cost  of  providing  additional  servlces-- 
from  the  improvement  in  the  quality  of  teaching  *nd  the  level  of  learning; 
from  the  ability  to  shrink  time  and  space,  and  from  the  sharing  of  limited 
resources,"  (4,  p.  56). 
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CAI  is  thereby  a  technology,  not  a  new  experimental  classroom 
technique.  Indeed,  the  magnitude  of  societal  effect  possible  with 
CAI  may  be  compared  without  any  hesitation,  to  that  of  the  industrial 
revolution.  The  potential  for  basic  improvements  in  educational 
systems  exists,  but  it  must  be  brought  to  the  proper  climax.  The 
economic  exploitation,  frustration,  disruption  of  employment ,  suicide, 
and  so  forth,  which  the  industrial  revolution  led  to,  could  have  been 
avoided  with  proper  planning.  With  the  advent  of  the  computer,  automa¬ 
tion  has  already  changed  man's  role  to  that  of  innovator  rather  than 
routine  performer.  On  a  purely  rational  basis,  therefore,  CAI  is 
worth  doing  wherever  individual  differences  make  a  difference  in  the 
field  of  instruction. 


CAI  Research  and  Development  is  Complex 

Having  answered,  all  too  briefly,  our  first  question  affirmatively — 
that  is,  CAI  ia  indeed  worth  doing— the  second  and  more  thorny  question 
must  be  addressed.  What  do  we  mean  by,  "It  should  be  done  right"? 

Figure  1  shows  Eight  Steps  that  comprise  a  total  systems  approach.  This 
framework  (with  slightly  different  words,  or  with  seven  versus  eight 
steps)  must  be  followed  from  research  through  development  and  implementa¬ 
tion  in  order  to  result  in  operationally  valid  and  useful  CAI  systems. 

The  Eight  Steps  of  the  Systems  Cycle 

1.  State  the  real  NEED  you  are  trying  to  satisfy. 

2.  Define  the  educational  OBJECTIVES  that  will  contribute  to  satisfying 
the  real  need. 

3-  Define  those  ‘real  world*  limiting  CONSTRAINTS  that  any  proposed 
system  must  satisfy. 

A.  Generate  many  different  ALTERNATIVE  systems. 

5-  SELECT  the  best  altemative(s)  by  careM  analysis. 

6.  IMPLEMENT  the  selected  altemative(s)  for  testing. 

7.  Perform  a  thorough  EVALUATION  of  the  experimental  system. 

8.  Based  on  experimental  and  "real  world*  results,  FEEDBACK  the  required 
MODIFICATIONS  and  continue  this  cycle  until  the  objectives  have  been 
attained. 

Figure  I 


A  proper  operational  CAI  system  must  pass  from  the  breadboard 
stage,  through  the  prototype,  to  an  operational,  cost/effective  phase. 
"Breadboard"  is  a  term  originated  by  electronic  circuit  designers. 

During  very  early  stages  of  developing  an  electric  circuit  the  paper 
design  (conception)  is  translated  into  a  set  of  components  provisionally 
connected  (by  alligator  clips  and  a  few  wires)  and  tacked  onto  a  wooden 
board.  The  purpose  is  to  verify  that  the  design  scheme  will  have  the 
general  characteristics  expected  of  it.  By  extension,  "breadboard"  refers 
to  any  first  and  provisional  realization  of  a  system  design. 


Applied  to  CAI,  documents  like  the  CED  (Committee  for  Economic 
Development,  July  1968,  2)  report  have  stated  the  need  in  education 
for  making  maximum  use  of  individualized  and  personalized  instruction. 

We  have  defined  our  objectives  in  terms  of  producing  student  output 
at  a  given  level  of  achievement.  As  part  of  the  design,  note  that 
this  process  is  to  be  an  iterative  one  in  attaining  that  goal.  In 
the  breadboard  iteration,  only  the  most  crucial  design  criteria  are 
applied.  Secondary  objpetives,  that  is,  desirable  features  or  "nice- 
to-have"  characteristics,  are  kept  from  confusing  the  basic  design 
problem.  For  example,  with  reference  to  CAI,  at  the  breadboard  stage 
it  is  inappropriate  to  consider  time-sharing  the  computer  with  batch¬ 
processing  operations . 

Similarly,  during  the  breadboard  stage  of  design,  operating 
constraints  are  minimized.  In  terms  of  CAI,  it  may  be  essential  to 
develop  inexpensive  student  terminals,  but  first  terminals  with  adequate 
characteristics  must  be  designed.  Further  limitations  stem  from  avail¬ 
able  computers  and  compatible  CAI  equipment  and  languages.  Current  CAI 
systems  are  divided  among  those  which  have  not  progressed  past  the 
breadboard  stage  and  those  which  have  tried  (unsuccessfully)  to  bypass 
this  stage.  Alternative  instructional  decision-making  strategies  and 
mixes  of  hardware  and  software  subsystems  with  selected  subject  matter 
must  be  considered.  Following  systematic  evaluation  and  study  of  the 
alternatives,  selection  of  an  initial  system  is  made  and  implemented 
for  a  test  run.  Evaluation  is  made  based  on  student  output.  Feedback 
to  improve  (modify)  iteratively  the  CAI  system,  namely,  meeting  objec¬ 
tives,  is  made. 

This  process  continues  throughout  the  iterative  development  in 
order  to  refine  the  system.  Once  the  breadboard  phase  has  been  com¬ 
pleted,  it  is  po.ssible  to  proceed  to  a  prototype  system  design,  the 
circumstances  under  which  the  system  must  perform.  In  this  phase  one 
first  establishes  precisely  what  the  system  is  to  do,  taking  into  con¬ 
sideration  major  constraints  such  as  permissible  costs  or  delivery  time. 
Various  available  means  (e.g.,  magnetic  or  optical  information  storage) 
are  weighed  against  optimization  criteria.  Optimization  means  a  best 
compromise  among  contradictory  objectives  and  imposed  constraints 
(e.g.,  lightweight,  portable  student  terminals  with  character-video- 
audio  display  capability  for  no  more  than  $500  per  unit)  in  terms  of 
some  ordered  se.  of  criteria  (price  more  important  than  display 
capability,  which  is  more  important  than  portability).  Finally,  the 
design  plan  that  has  emerged  is  implemented  and  a  first  prototype  is 
synthesized.  A  prototype  CAI  system  may  have  operational  usefulness, 
but  is  likely  to  include  design  flaws  and  oversights  that  ought  not 
be  multiplied  in  many  duplicated  installations.  A  prototype  is  merely 
an  untried  and  unadjusted  assemblage. 

A  tested  system  emerges  over  a  number  of  subsequent  repetitive 
development  cycles.  In  the  case  of  a  CAI  system,  only  a  small  number 
of  students  would  be  exposed  to  its  instruction  initially.  Their 
interactions  with  the  system  must  be  minutely  monitored  and  appropriate 
adjustments  made.  Massive  data  need  to  be  collected  in  order  to 
determine  whether  the  prototype  is  actually  performing  as  envisioned. 
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Where  actual  and  expected  performance  disagree  (e.g. ,  mean  delay  of 
system  response  to  student  exceeds  stipulated  value  of  one  second) 
revisions  must  be  made  in  the  system  design.  A  tested  system  exists 
only  after  observed  system  performance  coincides  with  expected  perform¬ 
ance.  Only  then  is  it  economically  justifiable  to  use  the  prototype 
design  as  a  template  for  multiple  reproductions. 

This  approach  is  also  necessary  when  attempting . improvements  in  a 
traditional  instructional  model.  We  shall  return  and  enlarge  upon 
this  approach  later  when  proposing  a  scheme  for  accomplishing  proper 
CAI  development.  The  significance  of  the  approach  is  amplified  ten¬ 
fold  with  an  innovation  as  encompassing  as  CAI  can  be.  The  framework 
then  does  not  preclude  CAI  as  drill  and  practice,  simulation,  or  use 
as  strictly  a  calculational  aid  for  problem  solving.  These  can  all 
be  useful  parts  of  an  overall  CAI  RgD  project.  However,  as  previously 
noted,  the  instructional  system  consists  of  N  components .  If  only  one 
or  two  components  are  clearly  identified,  measured,  and  controlled, 
their  contribution  to  instructional  output  relative  to  that  of  all 
the  N- 1  or  N-2  other  uncontrolled  components  becomes  extremely  diffi¬ 
cult,  if  not  impossible,  to  assess  (5). 

"Doing  it  right"  thus  means  having  a  careful  system  research  and 
development  effort  integrating  the  facets  of  computer  hardware  and 
software  with  content  development  and  studies  of  alternative  instruc¬ 
tional  strategies.  This  means,  in  turn,  a  multidisciplinary  effort. 

If  CAI  is  to  fulfill  the  promise  of  delineating  the  relative  importance 
of  the  various  characteristics  pertinent  to  appropriate  decision-making 
in  instruction,  it  must  have  the  opportunity  to  model  the  instructional 
situation,  to  vary  parameters,  and  to  apply  these  to  meaningful  human 
learning. 

The  Amount  of  Required  Effort  for  CAI  R&D  Is  Large 

Let's  consider  what  experience  to  date  has  shown  to  be  the  require¬ 
ments  of  large-scale  CAI  centers  for  annual  funding  levels.  From 
informal  discussions  with  knowledgeable  persons  regarding  their  CAI 
projects’  fiscal  problems,  it  appears  both  internationally1  and  in  the 
United  States  that  an  operating  budget  of  approximately  $250,000  to 
$300,000  a  year  is  necessary  merely  to  maintain  facilities  in  operations. 
The  reason  for  this  becomes  quite  clear  if  one  considers  simply  the 
rental  price  of  an  IBM  1500  system  as  an  example.  The  hardware  alone 
averages  $100,000  to  $110,000  per  year.  Taking  the  $250,000  total,  this 
leaves  $150,000  per  year  for  staffing,  administration,  and  support 
facilities.  The  result  is  that,  given  approximately  $38,000  cost  per 
professional  man-year,  one  can  have  only  a  minimal  CAI  program  (4  pro¬ 
fessionals)  concentrating  on  operational  activities.  If  research  is  to 
be  conducted  at  a  CAI  center  in  a  university  or  other  academic  institu¬ 
tion  (where  most  CAI  R$D  seems  to  be  taking  place),  it  must  depend 


1Mr.  Andre  Kirchberger,  Administrator,  Center  for  Educational 
Research  and  Innovation,  Organization  for  Economic  Cooperation  and 
Development  (OECD),  Paris,  France. 
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inevitably  upon  part-tim  participation  of  faculty,  students,  and 
ado.  listrative  personnel.  Most  of  these  individuals  have  other 
primary  responsibilities  demanding  the  major  part  of  their  energies 
and  attention. 

How  much,  then,  should  a  total  systems  approach  toward  the  develop¬ 
ment  of  CAI  require  to  produce  an  operational,  useful  example  of 
individualized  instructional  models?  We  cannot  answer  this  in  absolute 
terms,  but  on  the  basis  of  experience  with  Project  IMPACT'S1  multi¬ 
disciplinary  staff  involving  18  professionals  and  seven  others  in 
supporting  roles,  a  personnel  budget  of  at  least  twice  these  amounts 
seems  necessary.  Our  installation  is  unique  in  that  it  is  funded,  so 
far,  by  a  single  arm  of  the  government.  Of  course,  one  of  the  problems 
for  most  of  the  CAI  projects  is  that  they  are  funded  from  multiple 
sources  with  different  demands;  and  in  order  to  satisfy  the  require¬ 
ments  of  the  various  funding  agencies,  the  research  and  development  is 
fragmented.  Until  recently,  the  awareness  of  funding  at  a  "critical 
mass"  level  did  not  seem  to  exist.  Recently,  it  seems  that  the  U.S. 
Office  of  Education  has  taken  steps  to  support  fewer  projects  in  CAI 
but  at  a  higher  level  of  funding.  This  seems  to  be  a  move  in  the  right 
direction.  The  problem,  however,  in  maintaining  this  critical  level 
is  not  simply  to  permit  existence  of  an  operating  environment,  but 
rather  to  facilitate  large-scale,  integrated  centers  to  study  and 
arrive  at  both  the  adequate  descriptions  of  learning  processes  and 
the  necessary  prescriptions  for  instructional  development. 

A  recently  published  figure  from  the  Bureau  of  Research,  U.S. 

Office  of  Education  (6),  indicates  that  they  are  providing  an  average 
of  approximately  $287,000  per  project  over  10  CAI  projects.  If  other 
governmental  agencies  could  support  these  acane  projects,  without 
changing  their  goals  and  with  a  comparable  amount  of  money,  then  it 
would  seem  to  be  possible  to  proceed  beyond  the  breadboard  stage  of 
CAI  development. 

A  point  of  clarification  is  worth  noting  with  respect  to  encouraging 
flexibility  and  diversity  in  approaches.  The  advocacy  of  varying 
approaches  to  solving  the  strategy  development  and  overall  construction 
of  useful  CAI  should  not  be  in  any  way  misunderstood  as  the  support  of 
multiple  small-scale  efforts.  Furthermore,  if  a  funded  project  is 
required  to  continually  submit  and  resubmit  a  multiplicity  of  proposals, 
a  large  amount  of  time  and  effort  will  go  into  nonproductive  work.  A 
rational  basis  for  selection  of  project  proposals  should  be  used  at  the 
outset,  and  then  a  reasonable  amount  of  funding  be  provided  over  a 
period  of  at  least  five  years.  Without  advocating  that  all  the  eggs 
be  put  in  one  basket,  we  are  saying  that  the  highest  rate  of  progress 
is  likely  to  result  from  a  distribution  of  available  funds  to  fewer 
"baskets"  at  sufficient  levels  to  permit  large-scale  integrated  efforts. 
On  the  other  hand,  funding  a  diversity  of  projects  at  very  small  levels 
will  probably  result  in  none  of  the  efforts  providing  a  full-fledged 
operationally,  implementable  CAI  system  for  education. 


1HwRR0  Work  Unit  IMPACT,  Prototypes  of  Computerized  Training  for 
Army  Personnel. 
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RESOLVING  THE  PROBLEMS  OF  NECESSITY,  COMPLEXITY,  AND  MONEY 


An  Analogue  to  the  Corporate 
3t  to  5%  Investment  Is  Advocated 

What  expenditures  should  be  invested  in  a  CAI  total  systems  effort? 

We  can  start  by  taking  a  cue  from  a  corporate  model  and  note  that  a 
number  of  sources  (Duckworth  of  the  United  Kingdom,  7,  as  well  as 
representatives  of  corporations  in  the  United  States)  have  indicated 
a  3%  to  S%  level  of  corporate  income  is  appropriate  for  these  purposes. 

In  1960,  the  amount  spent  in  education  was  approximately  .1%  (Coombs,  8). 
Recently,  it  was  estimated  that  in  1968  the  entire  educational  enter¬ 
prise  in  the  United  States  cost  somewhere  around  $50  billion1.  The 
figure  for  expenditures  on  educational  research  now  reaches  approximately 
$100  million,  a  .2%  investment.  Perhaps  some  would  find  cause  for 
optimism  in  this  increase.  However,  even  with  a  large  error  factor,  it 
appears  "that  the  investment  in  educational  research  is  only  a  fraction 
of  1%  of  the  educational  enterprise."  In  contrast  to  this,  RSD  invest¬ 
ment  in  the  electrical  communications  industry  in  1968  stood  at 
approximately  3.4%.  IBM  reportedly  invested  $300  million,  or  roughly 
5%,  in  R§D  of  its  reported  $6  billion  gross  income  for  the  same  period. 

If  we  accept  the  3%  to  5%  figure  as  necessary  to  sustain  viable 
R&D  in  education,  the  annual  dollar  investment,  given  the  $50  billion 
total  year  expenditure,  should  have  been  between  $1.5  and  $2.5  billion. 
Before  you  cringe  at  the  apparent  enormity  of  this  amount,  consider  it 
relative  to  other  innovative  programs.  The  United  States  has  put  a  man 
on  the  moon.  The  United  States  has  invested  approximately  $24  billion 
and  10  years  of  research  and  development  to  go  from  the  breadboard 
stage  through  the  prototype  to  the  ultimate  system  for  the  lunar 
landing  and  retrieval.  Accepting  the  3%  to  5%  figure,  a  comparable 
time  period  for  innovative  developments  in  education  would  amount  to 
an  investment  between  $15  and  $25  billion.  We  need  to  consider  this 
question:  Is  education  any  less  valuable  than  a  lunar  landing  or  space 
travel  in  general?  Our  personal  reaction  is  that  both  efforts  are 
extremely  valuable. 

Let  us  carry  the  analogy  one  step  further.  The  dollar  cost  of 
investment  in  education,  in  particular  in  CAI,  further  pales  by  compari¬ 
son  to  the  costs  of  other  single  technological  developments.  For 
example,  the  U.S.  government's  contribution  so  far  to  the  controversial 
supersonic  transport  program  is  roughly  $1  billion.  The  cost  of  the 
deep-dive  nuclear  research  submarine  is  roughly  $99  million.  Many  more 
such  examples  could  be  cited.  Note  that  these  are  specific  system 
efforts.  We  have  previously  documented  that  an  implementable  computer- 
administered  instructional  system,  following  from  breadboard  through 
operational  development  stages,  could  be  accomplished  for  something 


Arthur  S.  Melmed,  "CAI— A  Five-Year  Perspective,"  a  symposium 
presentation  at  American  Educational  Research  Association  meeting 
(mimeographed  copy),  Los  Angeles,  February  1969. 
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in  the  order  of  $5  million  in  a  careful  time-phased  schedule  (Seidel,  9). 
We  submit  that  this  is  certainly  not  off  by  more  than  a  factor  of  two, 
and  further  that  the  value  of  such  a  development,  as  we  have  tried  to 
indicate,  can  be  monumental  in  reshaping  the  whole  field  of  education. 

Of  course  this  must  include  adequate  interdisciplinary  staffing  and 
evaluation  and  testing  of  all  the  components  of  experimental  hardware, 
computer  software,  instructional  strategy,  and  various  selected  subject 
matter.  However,  we  submit  again  that  the  expenditure  is  but  a  minute 
fraction  of  the  cost  for  other  technological  developments.  Surely  the 
goals  are  at  least  equally  desirable.  Also,  given  the  3%  to  5%  invest¬ 
ment  in  all  of  education,  enough  money  should  be  available  for  such 
innovative  efforts  as  CAI  to  permit,  as  Oettinger  has  proposed  (2),  a 
great  deal  of  flexibility  and  diversity  in  approaches.  It  thus  should 
be  possible  to  solve  the  educational  technological  problems  of  using 
the  computer  properly  in  the  entire  field  of  instructional  and  educa¬ 
tional  improvement. 

A  Proposed  Partnership:  Allocate  the  Unique  Resources 
Among  the  Steps  of  a  Systems  Approach 

It  seems  clear  that  we  are  faced  in  education  and  training  with  (a)  a 
commitment  to  unprecedented  financial  investment,  (b)  the  requirement 
that  this  effort  serve  the  public  welfare,  vis-a-vis  the  educational 
consumer,  as  opposed  to  a  private  corporate  entity  and  its  stockholders, 
and  (c)  the  recognition  and  acceptance  that  the  return  on  investment 
may  take  much  longer  for  accurate  calculation  than  do  industry's 
profit  and  loss  statements.  All  of  this  indicates  that  only  the 
Federal  government  can  adequately  and  properly  provide  the  funding 
(and  thereby  influence  the  policy  guidelines)  for  this  type  of 
partnership . 

Secondly,  the  techniques  of  industrial  research  and  development 
(i.e.,  product  development)  are  appropriate  to  contribute  to  an  evolving 
CAI  system.  Various  industries  have  the  facilities  and  the  tools  for 
development  and  production  (certainly  for  the  necessary  hardware  and 
software  components  in  CAI).  Non-profit  industry  employs  research 
personnel.  The  educational  community  is  a  unique  partner  in  the  sense 
that  the  local  school  system  and  its  personnel  are  going  to  be  the  focal 
point  of  the  development,  but  they  do  not  have  the  financial  resources 
nor  do  they  have  more  than  a  portion  of  the  total  personnel  requited  to 
accomplish  this  innovation  properly.  Their  partnership  contribution 
consists  in  developing  instructional  content  through  providing  subject- 
matter  experts  as  part  of  an  interdisciplinary  team,  and  by  providing 
the  demonstration  and  test  facility  within  which  the  innovative  experi¬ 
mentation  and  development  can  take  place. 

Finally,  in  addition  to  the  partnership  structure,  the  developing 
CAI  effort  must  include,  as  indicated  earlier,  a  functional  approach 
that  embodies  systems  design  and  total  systems  development  using  the 
eight  steps. 

To  place  the  partnership  within  this  approach,  the  initiation  of 
policy  and  guidelines  (statement  of  need )  would  come  from  a  central 
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government  agency.  (It  is  conceivable  that  in  a  somewhat  different 
form,  specific  needs  might  arise  from  statements  within  the  educational 
community.)  These  would  be  transformed  into  specific  system  obfeotivea 
jointly  by  team  members  representing  all  these  arms  of  the  partnership. 

An  interdisciplinary  research  and  development  effort  (comprising  the 
remaining  steps  except  evaluation  and  feedbaok)  would  be  conducted 
with  the  lead  role  most  probably  taken  by  a  non-profit  (rather  than 
profit- oriented)  R§D  corporation  (see  last  two  pages  for  reasoning). 

Local  government  and  educational  system  personnel  and  facilities 
would  provide  administrative  aid.  Industry  would  fabricate  the 
necessary  hardware  and  software,  and  members  of  the  three-way  team 
would  then  attempt  to  implement  the  provisional  system.  At  this  stage, 
the  work  primarily  would  be  accomplished  by  industry  and  the  educational 
community  with  administrative  aid  provided  by  local  government  (Figure  2) . 

Functioning  Model  in  the  Systems  Cycle 


IMPLEMENTATION 


Figure  2 


Considering  the  possibility  of  this  joint  venture  as  indicated,  one 
more  important  link  must  be  added.  To  provide  the  necessary  objective 
evaluation  and  feedbaok  in  a  coordinated  manner,  an  independent  fourth 
party  is  required.  The  form  of  the  feedback  information  would  be 
appropriate  to  the  particular  partner  of  the  three-way  team  in  order 
to  make  appropriate  modifications  to  those  aspects  of  the  systems 
development  process  uniquely  under  its  jurisdiction.  For  example,  if 
modifications  were  required  regarding  statements  of  need,  this  could 
be  provided  to  the  policy  and  guideline  process  for  the  government.  As 
the  modifications  were  necessary  in  fabrication  of  equipment  and  soft¬ 
ware,  it  would  be  given  in  unique  form  to  industry,  and  so  forth. 

While  this  may  be  difficult  to  conceive  of  within  a  given  country, 
it  may  not  be  far  off  from  what  is  currently  proposed  by  the  Organization 
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for  Economic  Cooperation  and  Development  (OECD),  an  international 
body  to  establish  policy  and  make  apolitical,  dispassionate  and  objec¬ 
tive  evaluations  of  CAI  studies.  In  the  United  States  fragmented 
examples  are  beginning  to  appear. 

PROBLEMS  TO  SOLVE 

Problem  of  Interpreting  the  Value  of  Education 

Now  we  shall  return  and  perhaps  answer  some  of  the  rejoinders  that 
may  have  developed  to  our  proposal  of  a  reasonable  level  of  dollar 
investment.  Let  us  consider  the  fact  that  the  goal  of  a  space  program 
such  as  putting  a  man  on  the  moon  and  retrieving  him  is  quite  specific. 

The  parameters  for  guidance,  for  control  of  the  entire  system  required 
to  perform  this  feat,  although  large,  are  finite,  measurable,  quanti¬ 
fiable.  A  major  problem  for  education  is  to  identify  all  those  factors 
that  are  pertinent  to  estimating  requirements  for  specified  achievement 
characteristics,  and  also  for  interpreting  the  value  of  that  educational 
achievement. 

Another  paramount  problem  that  educational  technology  faces  when 
contrasted  to  a  neat  and  closed  engineering  technology  such  as  the 
space  program,  is  that  the  return  on  investment  on  education  may  not 
be  visible  for  roughly  an  entire  generation.  The  attainment  of 
objectives  in  the  space  program,  on  the  other  hand,  was  relatively 
immediate,  dramatic,  quite  tangible,  and  rapid.  Our  only  rejoinder  is 
that  in  educational  technology  one  must  extrapolate  from  immediate  cri- 
terial  effects  of  measured  educational  proficiency  to  ultimate  criteria 
of  general  societal  significance.  Of  necessity  this  means  at  the  outset 
that  we  have  clearly  identified  the  parameters,  our  educational  inputs 
and  our  educational  outputs. 

We  must  not  permit  our  experience  with  the  ambiguities  and  vagaries 
of  the  traditional  educational  instructional  system  to  force  a  premature 
and  inappropriate  evaluation  of  computerized  instruction.  True,  there 
have  been  difficulties  in  measuring  a  "good"  teacher,  certainly  in 
traditional  instruction  one  of  the  most  significant  educational  inputs 
(Froomkin,  10).  But,  as  noted  previously  (Kopstein  and  Seidel,  1; 

Seidel,  9;  Seidel,  11),  in  computer- administered  instruction  we  can 
objectively  document  the  dimensions  of  that  system.  Cost  effectiveness 
oan  eventually  be  measured.  The  instructional  agent's  value  (instead  of 
the  human  teacher's)  oan  be  measured  against  its  output  (student  achievement). 
The  costs  of  the  entire  CAI  system,  input,  transformation  and  output,  oan 
be  evaluated  and  justified  or  not,  on  tangible  and  objective  bases. 

What  then  are  the  implications  for  consideration  of  cost  effective¬ 
ness  studies  in  CAI?  Granted  that  we  may  extrapolate  to  later  generation 
returns,  it  certainly  seems  we  must  consider  direot  outputs  of  the 
instructional  system  if  our  evaluation  is  to  have  any  substance  at  all. 
Standardized  achievement  units  must  be  the  criteria  (Randall  and 
Blaschke,  12).  Harley  (3,  p.  52)  in  an  insightful  appraisal  makes 
the  point  even  stronger.  "...  we  have  been  concerned  with  the  cost 
per  student  taught  (our  input)  and  not  the  cost  per  student  learned  (our 
output);  yet  we  know  that  what  is  taught  and  what  is  learned  are  not 
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synonymous."  We  have  ignored  therefore  the  vital  "...  cost  of  our 
scrap  (non-operable  units)— students  who  cannot  function  in  today's 
society  ..."  and  therefore  our  cost  statements  have  fallen  far  short 
of  the  true  costs  of  our  education  system. 

A  host  of  other  problems  beyond  the  scope  of  this  paper  relate  to 
establishing  other  measurable  values  of  education.  For  instance,  Suppes 
(13,  p.  12)  has  recently  called  for  "clearly  stated  normative  principles" 
for  dealing  with  the  contradictions  in  modem  philosophy  of  education. 
While  not  a  direct  concern  of  economics  and  CAI,  certainly  resolving 
the  existing  antonyms  of  educational  philosophy  (e.g.,  maximum  freedom 
of  choice  versus  development  of  a  sense  of  discipline,  content  versus 
method,  social  adjustment  versus  maximum  achievement)  is  essential  for 
a  meaningful  evaluation  of  any  educational  innovation.  This  is  par¬ 
ticularly  critical  (perhaps  propaedeutic)  for  CAI  which  depends  upon 
acceptance  to  a  large  degree  of  the  model  of  individualized  instruction. 
We  believe  that  it  is  an  extremely  important  problem  to  be  resolved. 

Problems  Exist  In  Establishing  a 
Potential  Partnership  Model' 

Another  item  is  equally  important  in  establishing  that  CAI  is 
valuable,  and  when  setting  the  necessary  dollar  figures  for  R&D  invest¬ 
ment.  That  is,  what  kind  of  a  workable  framework  can  exist,  within 
which  this  research,  development  and  operational  environment  is 
appropriate?  Can  industry,  government  and  the  educational  profession 
be  combined  to  bring  the  effort  to  fruition?  What  policies  must  be 
incorporated  to  bring  this  about?  How  can  resources,  money,  facilities, 
personnel,  and  time  be  most  appropriately  allocated? 

Are  Goals  Compatible? 

Before  delineating  the  proposed  prototype  for  the  three-way  partner¬ 
ship,  let  us  examine  potential  problems  that  may  arise  from  the  combina¬ 
tion  of  industry,  government  and  educational  professions  in  this  manner. 
One  can  ask  whether  the  model  of  research,  development,  and  operational 
utility  for  industry  is  compatible  either  with  the  educational  profession 
or  with  that  of  the  government.  Interestingly,  we  already  have  at  least 
one  failure  in  an  attempt  to  bring  these  three  entities  together. 

Project  ARISTOTLE  was  intended  to  be  a  catalyst  and  continuing  stimulant 
for  educational  innovation  through  these  three  arms,  and  it  has  been 
dropped  for  lack  of  support.  Perhaps  the  answer  lies  in  incompatible 
models,  industry  in  the  United  States  exists  to  manufacture  and  sell 
at  a  profit.  Its  investment  of  3%  to  5\  in  research  rests  clearly  in 
the  belief  that  the  return  will  occur  in  a  relatively  short  period, 
perhaps  two  to  three  years.  The  premise  becomes  muddied  when  this 
industrial  template  is  applied  to  education.  We  spoke  earlier  of  the 
generation  lag  for  evaluation  purposes  in  educational  products. 

Considering  the  educational  model,  the  goals  of  education  have  never 
been  defined  in  terms  of  profit  and  loss  statements.  Selling  a  product 
has  never  been  part  of  the  system.  In  fact,  a  frequent  criticism  in 
recent  years  has  teen  that  the  intellectual  aspects  of  education  were 
being  subverted  for  more  specific  and  practical  occupational  training. 
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It  seems  that  the  traditional  roles  of  industry  and  the  educational 
profession  are  so  antagonistic,  or  at  least  not  overlapping,  that 
throwing  than  together  without  creating  a  new  compatible  model  for  R§D 
purposes  in  CAI  is  to  create  an  anomaly  doomed  to  failure  at  conception. 
Moreover,  government  and  education  are  interested  in  welfare  of  the 
populace,  education  being— in  this  sense— a  subset  of  the  governmental 
function.  Industry  is  concerned  with  product  development  and  sales. 

But  need  these  be  different?  (Galbraith  said  in  The  Affluent  Society 
that  the  private  sector  could  not  handle  this  type  of  effort  on  its 
own,  but  why  not  a  partnership?) 

"Government,  Industry,  and  Education11 
Are  Not  Monolithic  Entitles 

The  so-called  education  "industry"  in  CAI  is  not  monolithic  and 
not  coordinated.  It  involves  hardware  manufacturers,  book  companies, 
and  computer  software  houses  at  the  very  least.  The  companies  are 
relatively  independent  with  their  own  profit  and  loss  statements, 
corporate  policies,  and  so  forth,  and  they  all  must  be  brought  together 
in  order  to  construct  a  meaningful  CAI  system.  The  educational  community 
and  the  government  are  also  made  up  of  components  with  differing  capa¬ 
bilities.  "Educational  community"  consists  of  at  least  administrators, 
middle  management  in  the  person  of  assistant  principals  (or  assistant 
headmasters)  and  the  teachers  on  the  one  hand.  On  the  other  hand  is 
the  ultimate  user,  the  student.  Government  is  represented  by  central 
and  local  spheres.  For  funding,  we  require  federal  assistance  and  for 
participation  in  implementing  CAI,  we  require  local  involvement. 

There  may  well  be  many  objections  to  a  proposed  cooperative  set  of 
overlapping  functional  relationships  among  industry,  government,  and 
the  educational  community.  Not  the  least  of  these  is  an  abhorrence  at 
attempting  to  apply  the  corporate  model  of  R§D  to  education.  But  we  ask 
the  question:  Why  not?  It  has  produced  inventiveness,  increased 
profits  and  viable,  new  products  in  the  marketplace.  Why  not  ask  the 
same  of  education  R§D? 

Will  the  corporate  model  apply?  If  education  is  to  be  considered  as 
is  industry,  it  is  peculiar  in  that  the  financial  resources  are  clearly 
not  within  the  system  per  se,  but  must  be  provided  by  outside  sources — 
the  local,  regional,  and  central  governments.  Recently,  this  has  been 
called  into  question  by  those  who  would  like  to  apply  the  corporate 
model  (13);  but  the  alternatives  are  not  being  readily  accepted  (they 
imply  clear,  behavioral  definitions  of  educational  objectives,  etc.)  by 
the  educational  community. 

This  difficulty  is  epitomized  in  a  statement  by  a  noted  special 
assistant  on  education  who  was  in  the  former  presidential  administra¬ 
tion — "Perhaps  the  most  traditional  .  .  .  ,  and  the  one  (local 
community)  most  resistant  to  outside  change  has  been  the  educational 
community"  (14).  The  purpose  of  the  Elementary  and  Secondary  Education 
Act  of  1965,  Title  III  program,  is  to  provide  federal  grants  directly 
to  local  schools  "...  for  the  very  purpose  of  stimulating  innova¬ 
tion  and  change  in  local  educational  patterns."  The  prevailing  mood, 
however,  has  been  deep-seated  suspicion  in  the  United  States  that 
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federal  aid  means  federal  control.  Consequently,  a  serious  question 
is:  Even  given  sufficient  funding  resources,  how  can  a  workable  coopera¬ 
tive  model  be  developed  to  ensure  (a)  valid  R§D  involving  the  educational 
community,  and  (b)  proper  implementation  in  local  systems.  Part  of  the 
solution  must  of  neoeeeity  involve  oormitment  of  dollar  and  personnel 
reoouroee  toward  large- eaale  RSD  efforts  in  CAI  and  toward  massive 
training  and  retraining  in  the  educational  commnity. 

Let  us  turn  the  question  around  and  ask:  Is  education  becoming  an 
industry?  Should  it  be?  The  problem  would  seem  to  be  one  of  retaining 
the  goals  of  intellectual  expansion,  freedom,  and  innovation,  while 
adopting  where  feasible,  a  model  of  improvement  in  education  drawing 
upon  the  techniques  of  industrial  development.  What  also  seems  to  be 
required  at  the  very  least  are  policy  guidelines  from  the  federal 
government  to  force  a  necessary  workable  structure.  Assuming  that  the 
policy  is  appropriate  to  the  task,  this  would  aid  not  only  industry, 
but  the  educational  profession  and  the  educational  user--the  student. 

A  Vehicle  for  CAI  Research,  Development, 
and  Implementation  Is  Proposed 

A  definite  means  by  which  the  proposed  partnership  could  proceed 
efficiently  is  the  last  item  we  will  discuss.  Within  the  United  States, 
there  are  two  such  vehicles.  We  propose  first  the  establishment  of  a 
national  center  for  research  and  development  on  innovation  in  educational 
systems.  (We  compliment  the  Center  for  Educational  Research  and  Innova¬ 
tion  that  now  exists  in  France.)  We  think  it  is  necessary  to  establish 
such  an  institution.  In  fact,  this  has  been  proposed  previously  for 
the  U.S.  Navy  Department,  but  was  not  funded  by  the  Department  of 
Defense.  The  Center's  prime  function  would  be  to  coordinate  the  appli¬ 
cation  of  diverse  scientific  and  technological  principles  in  the  solution 
of  educational  problems  and,  generally,  to  evolve  educational  technology 
to  higher  and  higher  levels.  At  any  rate,  this  center  would  have 
closely  tied  regional  satellites  to  carry  out  the  translation  of  the 
results  of  the  research  and  development  into  operational  reality  within 
local  school  systems.  The  fourth  element,  evaluative  entity,  described 
earlier,  fits  here.  It  would  be  this  entity  that  would  bring  together 
the  industry,  central  government  funds,  and  local  governmental  and 
educational  systems  for  reorienting  personnel  into  a  demonstration 
program. 

At  the  national  center  a  training  emphasis  would  be  given  to  pre- 
and  post-doctoral  levels  in  order  to  develop  increased  national 
competence  in  instruction  theory.  The  areas  to  be  included  might  be 
illustrated  by  Computer  Sciences,  Behavioral  Sciences,  Applied 
Mathematics,  in  general  what  we  might  call  educational-technological 
research,  technical  writing  programs,  and  so  forth.  On  the  regional 
level,  the  regional  centers  would  train  for  tusks  relating  to  the 
inqplementation  of  CAI,  that  is  orienting  communities,  training  local 
officials,  local  administrators  of  programs,  and  so  forth.  It  is 
important  to  make  clear  that  the  national  R8D  Center  will  not  be  the 
sole  technical,  or  active  research,  installation.  Research  and 
development  would  be  carried  on  at  local  installations  to  permit 
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flexibility  in  approaches  and  there  would  be  technical  feedback  or  input 
in  both  directions  to  upgrade  the  RftD  status  in  general.  A  plan  for 
coordinated  activities  among  the  regional  and  the  national  centers  would 
be  essential  to  avoid  f  ragae.it a ti on  and  lost  efforts. 

To  accomplish  testing  and  implementation  there  already  exists  a 
potential  prototype  of  a  fourth  party  in  the  United  States.  It  is  a 
nonprofit  corporation  (the  Institute  for  Politics  and  Planning)  which 
has  the  role  of  bringing  together  government,  industry,  and  educational 
systems.  This  takes  the  form,  for  example,  of  establishing  advanced 
learning  centers  in  conjunction  with  the  government,  in  this  case  not 
only  the  central  government  as  the  funding  source  but  the  local  govern¬ 
mental  groups  as  well  as  the  local  educational  system  participating  as 
members  of  a  testing  and  evaluation  team.  (Currently  the  profit-minded 
industry  operates  on  a  fixed-fee  basis,  and  gets  paid  if,  and  only  if, 
students  succeed  in  reaching  established  achievement  criteria.) 

This  effort  is  somewhat  premature  with  respect  to  CAI  because  it 
assumes  that  we  are  now  able  to  accooplish  CAI  implementation  on  a 
large  scale.  Nevertheless,  a  nonprofit  corporation  may  well  be  the 
focal  point  for  making  a  partnership  viable  regarding  all  the  activity 
phases,  research,  development,  and  implementation.  In  the  nonprofit 
corporate  model,  goals  tend  to  be  more  directly  oriented  toward  the 
welfare  of  society  than  those  of  the  profit  maker  who  must  put  survival 
first.  Alternatively,  it  may  be  possible  to  adjust  the  goals  of  the 
profit  corporation.  If  the  return  on  investment  cycle  were  extended 
(to  five  or  more  years)  through  the  provision  of  some  type  of  government 
"insurance,"  profit-making  industry  could  play  a  role  equivalent  to  the 
nonprofit  entity.  The  "insurance"  debt  could  be  retired  at  the  end  of 
some  agreed-upon  period  either  out  of  the  corporation's  assets  (if  no 
marketable  product  was  delivered)  or  as  a  percentage  of  the  pro "Its 
derived  from  an  iaplemented  product.  This  at  least  represents  one 
mechanism  to  consider. 

In  any  event,  the  partnership  model  would  have  to  encompass  all 
the  facets  of  research,  development,  and  imp lamentation.  Too  frequently, 
an  innovation  is  developed  and  implementation  is  given  short  shrift  in 
the  form  of  simply  a  written  recommendation.  To  quote  Mr.  S.  Clark 
Beise  (Chairman,  Executive  Committee,  Bank  of  America)  in  the  report 
on  innovation  in  education  from  the  Committee  for  Economic  Development, 
"One  of  the  major  problems  inhibiting  change  in  our  present  educational 
programs  and  processes  is  the  lack  of  communication  between  educators, 
teasers,  administrators,  school  boards,  and  the  public. "I  He  goes  on 
to  note  that  the  statement  in  the  report  by  the  CED  develops  a  program 
that  should  be  accomplished  for  research  in  innovation  but  does  not 
carry  the  recommendation  through  sufficiently  "to  the  point  of  being 


^latt,  a  systems  analyst,  adds  the  dimension  of  unawareness 
"  ...  of  the  investment  aspects  of  education  ..."  to  the  problem. 

He  describes  the  public  and  private  school  system  as  a  "  .  .  .  network 
with  very  few  nodes  sufficiently  concentrated  to  allow  one  to  enter  and 
perturb  the  system."  (IS,  p.  417)  It  seems  our  task,  then,  is  to  create 
the  requisite  nodes'. 
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able  to  demonstrate  their  value  to  those  who  must  be  convinced  that 
changes  should  be  made.  In  order  to  disseminate  information  on  recom¬ 
mended.  changes  effectively,  there  should  be  established  a  system  of 
demonstration  schools,  reasonably  available  geographically ,  to  show 
what  can  be  done  in  general  practice  to  implement  and  integrate  the 
reaomended  improvements ,  within  practical  costs,  into  a  rounded 
program."  (Italics  added.) 

Ww  trust  also  that  this  discussion  has  demonstrated  that  "CAI  is 
worth  doing,"  and  that  the  approach  proposed  herein  has  suggested  a 
reasonable  framework  for  "doing  it  right."  We  are  certain  that  alterna¬ 
tive  proposals  can  and  will  be  made  to  deal  with  the  complex  problems 
of  resource  allocation  for  CAI  research,  development,  and  implementation. 
The  rough  sketch  given  in  this  paper  can  be  viewed  as  no  more  than  an 
opening  wedge.  However,  we  are  equally  certain  that  some  such  approach 
must  be  put  forward  and  developed  as  a  workable  model  if  CAI  is  to  live 
and  fulfill  its  vast  potential. 

Although  we  have  been  concerned  with  the  field  of  education,  this 
discussion  generally  applies  to  training  and  CAI  also.  To  the  extent 
that  training  can  represent  a  more  well  defined  '  ibset  of  instructional 
processes  (e.g.,  objectives  and  achievement  measures)  then  the  task 
of  moving  from  research  through  development  to  operational  CAI  is 
made  easier.  But  the  steps  are  the  same,  the  required  commitment  is 
firm,  and  the  goals  are  still  waiting  to  be  achieved. 
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